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This paper reports a process of comptiter -processing tests 
which the writer has used to some advantage in an instructional 
situation where weekly multiple -choice testing was cdvisablSo 
The process may be useful to others either to meet a similar 
instructional need or for other reasons o The report Is brief 
but further details can be furnished on request c Details 
concerning luachine processing and computer progranaalng heve 
been held to a minimtsm., sine : they would not foe useful to vhe 
typical reader o 

The instructional problem which caused the author to 
pursue a new testing process was the followings Students in 
Political Science (American Government) are disposed to take 
the course lightly since it Is not usually associated with a 



program and for other reasons o This means that they 
do not prepare adequately for class on a day-to-day lasiso 
As a result p they miss what Is perhaps the most valuable part 
of the course - the continuing training in investigation ofp and 
discourse about political problems ^ This is training which 



is most vital to the pursuit of Intelligent adult citizenshipo 




For training of this type to be effective,-, the student should 
feel the urgency of preparing much as he would for a language 

class o Except for the necessity to impart this trainingi, there 
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would be no objection to any study in.etb.od. which the student 
wanted to foUoWo That is to say- if the objective wesre simply 
to impart information about the mechanics of government., it would 
not really matter whether this information was acquired all in 
one night before the final examination t?r even before the coux'^se 



had started - 

in accordance with this tralning->in=^ discourse concept of 



'feliG ^"unctiLcn 

were made essays in which the student was challenged to convince 
scmeone else of his view on an important political matter-, To 
prepare for thxs,- the students were rewarded well, for parties, 
pation in class discussion and debate- They were not rewairded 
merely for o3.ass attendance ©r for part texpaa. sun,- however sxnce 
the former was consistent with passivity and the latter was very 
difficult to evaluate. Instead, they were rewarded for preparing 
for class dls*^x3ssion and debate on the aasiMption that actual 



participation wo7xld follow, 
which were not descriptive 
readi?igs of this character 



Students were assigned materials 
blit fic^inkly argmentative. When 
are assigned. It is not too difficult 



to test proper cc®iprehenBioTi of them., as is illustrated by 



the 



following example s 

The first chapter of your reading XI Z takes the ^ position 
that all redheaded men arc warlsdcco Th.8.s position Is 
supported 

1- Statements showing that no folackheaded men 
are warlike • 

2- Examples of redheaded warlike men 

3- Arguments froin genetics 

Nokis of Tlbji!?. . svticfi posL^xen i.<s Yioi- 

taken 



Another eswiiaple ’would bej. 

Of the men you have read this week- wbi.cb would, be likely 
to favor Medicare for- the reason that it would correct 
a social injustice caused by tRe industrial revolution; 

In Hamilton 
2o Madison 
3o Jay 
Paine 

Such quest ions o if given f requently on assigned readings r- 
encourage students to read properly and ccntlnuouslyo This 
in itself is valuable training . It also results in improved 
classroom part Icipat ion ^ s.ince students are better prepared.^ 
although the desirable 100% participation level is certainly 
not achieved by this methods Participation based upon this 
kind of preparation is of greater advantage to the student than 
mere recitation^ since there is greater incidence of the position- 
talcing and position defending which is a necessas^y characteristic 
of true discussion and arg^rnientatioiio 

It was thought deei.rable to emphasise the value of continuous 
high level perfomanee and tc keep reminding students of their 
progress on a cumulative basis Therefore o when the second test 
was given it was to be reported back with the fi.rst test and 
In fact scored together with the first test.-, Consequently 
students were graded on the ba»is of percent right of the total 
taken to date- If,, for example, on a first five -point quls,. a 
student goe four right*, he received a score of 80,. If,, on the 
second five point test. he. got only t:M*ee r.lght, he rece.ived 



a score of 70 \7 out of 10 not 60 (3 aut of S)o 



And so on^ 



Xt was also considered advisabl.e to report 
to each student along with his own 



back correct ansx^ers 
,s in this wayo the 



student could check his work against the test which he had been 
allowed to' keep , and could leat^n fi'-om his own mistalceso 

IBM cards are used to collect answers to objective quest ions o 
Blank cards are distributed to students along with electrographic 



pencils,- Each card contains spaces in which to mark a student 
number and other spaces in which to icarlc answers to almost iOO 
questioner To indicate his student number ^ the student draws 
lines across five '’bubbles'^ which surround digits of the student 
number,-. To answer a quest ion^, he drawn a line across a “'bubble'"^ 
which contains the msmber cf the answei' which he has selected^ 
One advantage of the use i'.^f cai'ds Is thAt all processing is 
automatic^ From the ’"bubble" markings ulcnef. data processing 
equipiaent can ideifxtify the student,-, grace his paper and issue 
a report by name or by student ntmifoer on cards i one per student ) 
or or. a listing • ^riie grading is accomplished by comparing 
student answers with those indicated by thr. instructor on a 
specially designated master card-. The data processing equip=- 
ment is able to "‘remember^’ the students previous performance 
and to report his present score on the above -ouv lined cumulative 
basis. This abil.lty of data processing equipment to “’remember*" 
previous performance also raakes it possible o if the instructor 
so clesires;, to ke.ep a pez’formance record for each student (or 
duplicate performance records) in a centrally locat.ec data 
processing file. This file would be available in case the 



iiistrtictor’ 3 records wre lest' Ir. could also be ased to auto 

(laticalXy compute a final course grade cn the. basis of weighting 

^ r other instructions from the instructor In other words?, the 

-:inal course grade could be assigned by data processing with the 

■'«.nal escamlnation forming one element and prior performance 

¥ 

>,/’Gvidlng other slemei\ts.. 

One advantage then?, which a card answer form has over a 



i-=^per answer form Is that it can be processed and resu3.ts can be 



■ticc simulated with previous results all automatic^ ly wf.th the 
-p -tmetor doing no more than making cut the tests and marking 
orrect -answer card^, But this is not the only,, or even the 
ij r advantage- The card is al-^so much more easily manipulated 
■ purposes of test analysis- One analysis will, give an in- 



Htlon of the efficleviey of a quest ion in selecting between 



• . d and poor students- Usirig the overall rest scores., the machine 
-■^7-.ks sUidenta and places them in quart lies or deciles, A 

lectlve questiori will be one where higher deciles or quart lies 
,now more correct answers than do Slower ones., A convenient 
ndej5€ can be calculated for rating questions on this basis ^ 

Jne such inde:jt: is calculated by taking the proportion of right 
answers to total achieved by the lowest quart He stud^ants 

and subtracting this fr«a the propc.<'xion oi right answers to* 
total answers achieved by th« higher>i quartile students, Thusr, 
if 5 eat of 10 situdejvirr. ch(.' right an<;4wer in the highest 
quartiXe ana 3 cat of lO in -.he 3ot>--iac. . then the calculation wculd 



01 



5 

to 



3 

To 



««wt 

lO The r:.-fev?i.t i& ill ways .a number between plus 



1 and minus I with xiegatlx?e ntombers indicatiixg that the question 
tends to benefit poor students more than good student s« A 



Tupplement to this indeM: can foe usefully calculated* It shows tb. 
access of the question in terms of how difficult it wfijs.-, If 

> '•Vknir^v* O TT>1 A rt*?' •? *F Irnrb mflitw cTfti: xi” 

f AiULiVib O «« V W W ^ ^ ^ — •*" *--*/ — - ” •*■ 

/■IghtJ' it does not do the job of discriminati-ng between good and 
?^ad students which It should dOr Selectivity and difficulty 



' ade^ces when used together provide the instructor with a good 
indication of the usefulness of each individual item on his 
This evaluation can lead to changing sxibsequent testa for the 
be ter or it can be used to adjust the grading of the test for 
v’^-^ch the indexes are beltig calculated.-, The instructor can 
<xy'-zi£y that certain types of questions Csuch as those where 
■‘ e than thirty percent or more than ninety percent of student - 
u.';> > the right answers.) be eliitilnated automatically before final 
="v-’des are assigned for the test« 

Another useful set of analyses? can be obtained based on 
•.*?'itent analysis of each taultiple- choice questioTio Frequently,, 
r<> It iple -choice questions are fomixiated on the basis cf an 
•assumption that one answer is best., another is almost as good,, 
c' third Is likely tc look attx'active to a person who has no 
•. amiliarity with the oaterial at allf. and a foux'th is only for 
the random guessersr Such questions are good... since they engage 
che instrixctor in an analysis of how the material which he is 
ceachlixg is affecting the students., and why stiidents are likely 
to make errors on quest inis . Whet: he. r or not a question was 



•'*€iCiiivGd 3.1CI irtiC ?.XXtrG.T\d.CI.Ci S ifl Om 3 £‘i- 6*c}^sYicy 

oLint showing how many student s get each answet' and how many 
v^udents in each quartile cr decile got each answer ^ Such an 
-I’alysis may sbow,^ for eijcample that the very best students 
•«i the class are answering the second, best answer, The reasons 
such a finding wo^iXd be well worth Inveoitigetingo Also, a 
index (a coefficient of correlation; can be calculated 
m^..asure the insfcn^ctor s success in predicting which answer 



6 .he best, second best«^ thix*ci best, and xoiirth bejat'. 

These are some analyses of test answers which the author 



^‘sed. 



and which were made possible by using card answer 



’r '•?>' rather than the more usxT.a.1. paper answer tonnS " Other 
du ' ' 'v ses are also pcrssible- For example, a computer can scan 

I les of student paper;^ for pattern.’^ of m-ong answers in such 
^ as to check for stwient cheating • .ihe pattern of answer 
.v‘'. ' ' ysis groups students who have ai..l or most answets the sameo 
h-.> permitting the instructor to intelligently p^irsuo. the 
: sl'ion of possible cheating' The results of these analyses 



. uld assist the teacher materially 
and helpful testjs whi.le unburdening 
anical routine associated with (cest 



in administering frequent 
him from much of the mech 
analysis.;, grading and record- 



ing grades. 



